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Abstract 

We report on searches for B~ lP K~ and B l)*"K~ , with vr'^ or 7, and 

^D^ ^ K^TT^ (and charge conjugates). These final states, which we denote as [K~^7r~]DK~ and 
[K~^TT'^]£)*K~ , can be reached through the b ^ c transition B~ — > D^*^^K~ followed by the doubly 
Cabibbo-suppressed — > K~^tt~ , or through the b ^ u transition B~ D^*^^K~ followed by the 
Cabibbo-favored — > K^ir^, or through interference of the two. Our results are based on 227 
million T{4S) — > BB decays collected with the BABAR detector at SLAG. We measure the ratios 



r{B+ ^ [K-7r+]DK+)+r{B- ^ [K+7t-]dK-) _ ..on 



and 



_ r{B+ ^ [K-7r+]n^^D^oK+)+TiB- ^ [K+7T~]n,^o^oK-) _ ...,,o.oio 
^ = r(i?+ ^ [K^.-]^,^^^oK^) + T{B- [K-.^]^,^^^oK-) " "O-OO^^o.ooe 

^ ^'^•^^ ^ r(i?+ ^ [K+vr-],,.^^,ir+) + r(B- ^ [i^-vr+]^.^,,,K-) - • 

We set TZktt < 0.030 (90% C.L.) and from this limit we extract the amplitude ratio rs = \A{B~ — > 
D^K-)/A{B- D^K-)\ < 0.23 at the 90% confidence level. This limit is obtained with the most 
conservative assumptions on the values of the CKM angle 7 and the strong phases in the B and D 
decay amplitudes. From the measurements of Ti* Ktt,d-k° ^"^^ T^* K-n,D'y we extract r*^ = \A{B~ 
D°K-)/A{B- D^K-)\'^ = 4.61^^3^ ^ 10"^ and r*^ < (0.16)2 at the 90% confidence level. 
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1 INTRODUCTION 



Following the discovery of CP violation in i?-meson decays and the measurement of the angle /? of 
the unitarity triangle [1] associated with the Cabibbo-Kobayashi-Maskawa (CKM) quark mixing 
matrix, focus has turned towards the measurements of the other angles a and 7. The angle 7 
is &^Ei~^ub^ud/^cb^cd)^ where V^j are CKM matrix elements; in the Wolfenstein convention [2], 
7 = arg(K*fe). 

Several proposed methods for measuring 7 exploit the interference between — > 
and B- (Fig. 1) which occurs when the D^*^^ and the D^*^^ decay to common final 

states, as first suggested in Ref. [3]. 




Figure 1: Feynman diagrams for B~ d'^*^^K^*^~ and The latter is CKM- and 

color-suppressed with respect to the former. The CKM and color suppression factors are expected 
to be roughly | KifeKs/^cfeKTsI ~ 0-4 and 1/3 respectively. 



As proposed in Ref. [4], we search for B' K~ and B' y*^K-, ^)*^^ 'nK'^f-f followed 

by — > K^ir^ , as well as the charge conjugate sequences. In these processes, the favored B decay 
followed by the doubly CKM-suppressed D decay interferes with the suppressed B decay followed 
by the CKM-favored D decay. The interference of the b ^ c and b ^ u amplitudes is sensitive to 
the relative weak phase 7 = arg{-VudV*ij/VcdV*^). 

We use the notation B^ [h^ h2]Dh^ (with each hi = tt 01 K) for the decay chain B^ — > i^^/ig , 

hih2- For the closely related modes with a i)**^ we use the same notation with the subscript 
D replaced by D*. We also refer to /13 as the bachelor tt or K. 

In the decays of interest, the sign of the bachelor kaon is opposite to that of the kaon from 
D decay. It is convenient to define ratios of rates between these decays and the similar decays 
where the two kaons have the same sign. The decays with same-sign kaons have much higher rate 
and proceed almost exclusively through the CKM-favored and color favored B transition, followed 
by the Cabibbo-favored -D-decay. The advantage in taking ratios is that most theoretical and 
experimental uncertainties cancel. Thus, ignoring the possible effects of D mixing and taking into 
account the effective strong phase difference of tt between the D* decays in D'j and D7r'^[5], we 
define the charge-specific ratios for D and D* as: 



'^^^ " T{\K^7:^]],K^) ='B + ri + 2rBru cos(±7 + 6) (1) 
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and 



where 



' r([K±7rT]D«_D7i^±) 

A{B- 



r*i + r|) + 2r*BrD cos(±7 + 5*) and 
rB 2rBrD cos(±7 + 6*), 

D^K-) 



A{B- D^K-) 

A{B- — D''^K~] 



TD = 



A{B- D*^K-) 



A{D^ K-K+) 



0.060 ±0.003 [6], 



(2) 
(3) 

(4) 

(5) 

(6) 
(7) 



and 5^ and 5d are strong phase differences between the two B and D decay amplitudes, respec- 
tively. 

We also define the charge-integrated ratios: 



T{B- [K+7r-]DK-) + r{B+ [K-7r+]DK+) 
r{B- [K-tt+]dK-) + r{B+ [K+Tr-]DK+) 



and 



7^ 



Kn,Dn°{Df) — 



T{B- [K+TT-] 



D* 



K-) + T(B+ [K-TT^ 



jD*^D7rO(D7) 



Then, 



K-K 



= + r|) + 2rBrD cos 7 cos 5 



K'K,D-K° 



and 



K7r,D'y 



r*Q + r^) + 2r^rD COS 7 COS (5*, 



f*B + ''d ~ 2r*QrD cos 7 cos 5* , 



(8) 

(9) 

(10) 
(11) 

(12) 



assuming no CP violation in the normalization modes [K^7r^]£)K^ and [K^Tr^jD^K^. In the 
following we use the notation B."}^^ when there is no need to refer specifically to R 



(*) 

Since r^' is expected to be of the same order as rjj, CP violation could manifest itself as a large 

difference between the charge-specific ratios TZ^^ and TZ^^ ■ Measurements of these six ratios can 

be used to constrain 7. 

(*) 

The value of r^^ determines, in part, the level of interference between the diagrams of Fig. 1. In 

(*) 

most techniques for measuring 7, high values of lead to larger interference and better sensitivity 
to 7. As we will describe below, the measured TZ)^^ are consistent with zero in the current analysis. 
This allows us to set restrictive upper limits on since TZ^xn depend quadratically on 
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In the Standard Model, r^^ = \V^bV^/V^V*^\Fcs ~ OAF^s- The color -suppression factor F^g < 1 
accounts for the additional suppression, beyond that due to CKM factors, of B~ D^*^^K~ 
relative to —>■ D^*^^K~ . Naively, F^s = \, which is the probability for the color of the quarks 
from the virtual W in B~ — > D^*^^K~ to match that of the other two quarks; see Fig. 1. Early 
estimates [7] of F^s were based on factorization and the then available experimental information on 
a number of 6 — > c transitions. These estimates gave Fes ~ 0.22, leading to « 0.09. However, 
the recent observations and measurements [8] of color suppressed 6 — c decays {B D^*^hP; 

= 'rr^,p^,uj,rj,ri') suggest that Fes, and therefore could be larger. 

In this paper we report on an update of our previous analysis of B~ K~ [9], and the first 

attempt to study B~ — > ^*^K~ . The previous analysis was based on a sample of S-meson decays 
a factor of 1.9 smaller than used here, and resulted in an upper limit TZk-k < 0.026 at the 90% 
confidence level. This in turn was translated into a limit tb < 0.22, also at 90% C.L.. On the other 

hand, a study by the Belle collaboration [10] of B^ ^^K^ and B^ i^*^K^, &^ K^tt+tt", 
favors rather large color suppressed amplitudes: rs = 0.26lo and = 0.201q;^8. 

2 THE BABAR DATASET 

The results presented in this paper are based on 227 x 10® T{AS) BB decays, corresponding 
to an integrated luminosity of 205 fb~^. The data were collected between 1999 and 2004 with the 
BABAR detector [11] at the PEP-II B Factory at SLAG [12]. In addition, a 16 fb"^ off-resonance 
data sample, with center-of-mass (CM) energy 40 MeV below the T{4S) resonance, is used to study 
backgrounds from continuum events, e"'"e~ qq {q = u,d,s, or c). 

3 ANALYSIS METHOD 

This work is an extension of our analysis from Rcf. [9], which resulted in limits on TZktt < 0.026 
and rB < 0.22, as mentioned above. The main changes in the analysis arc the following: 

• The size of the dataset is increased from 120 to 227 x 10® T{AS) ^ BB decays. 

• This analysis also includes the B"^ —>■ mode. 

• The analysis requirements have been tightened in order to reduce backgrounds further. 

• A few of the requirements in the previous analysis resulted in small differences in the efficien- 
cies of the signal mode B^ —>■ [K^7r^]K^ and the normalization mode B^ [K^tt^]K^ . 
These requirements have now been removed. 

The analysis makes use of several samples from different decay modes. Throughout the following 
discussion we will refer to these modes using abbreviations that are summarized in Table 1. 

The event selection is developed from studies of simulated BB and continuum events, and off- 
resonance data. A large on-resonance control sample of Dtt and D*Tr events is used to validate 
several aspects of the simulation and analysis procedure. 

The analysis strategy is the following: 

1. The goal is to measure or set limits on the charge-integrated ratios TZr-k and T^kt^- 
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Table 1: Notation used in the text. 



Abbreviation 


Mode 


Comments 


DK 


B- 


D^K-, K-TT+ and c.c. 






normalization mode 


Dtt 


B- 


D^TT-, L»o K-7r+ and c.c. 






control mode 


DK 


B- 


^^K-, K+TT- and c.c. 






signal mode 


D*K 


B- 


D*°K-, ^ DV/7, - 


> K-TT^ 


' and c.c. 


normalization mode 


D*Tr 


B- 




K-Tr+ 


and c.c. 


control mode 


D*K 


B- 




K+TT- 


and c.c. 


signal mode 



2. The first step consists in the application of a set of basic requirements to select possible 
candidate events, see Section 3.1. 

3. After the basic requirements, the backgrounds are dominantly from continuum. These are sig- 
nificantly reduced using a neural network designed to distinguish between BB and continuum 
events, see Section 3.2. 

4. After the neural network requirement, events are characterized by two kinematical variables 
that are customarily used when reconstructing B-meson decays at the T{AS). These vari- 
ables are the energy-substituted mass, mss = + " Pb)'^/Eq — p\ and energy difference 

A.E = E'^ — \\/s-, where E and p arc energy and momentum, the asterisk denotes the CM 
frame, the subscripts and B refer to the T[AS) and B candidate, respectively, and s is the 
square of the CM energy. For signal events mss = rns and AE = within the resolution of 
about 2.5 and 20 MeV respectively (here ms is the known B mass). 

5. We then perform simultaneous fits to the final signal samples {DK and D*K), the normal- 
ization samples {DK and D*K), and the control samples {Dtt and D*Tr) to extract TZktt and 
TIktt^ Section 3.3. The fits are based on the reconstructed values of mss and AE in the 
various event samples. 

6. Throughout the whole analysis chain, care is taken to treat the signal, normalization, and 
control samples in a consistent manner. 



3.1 Basic Requirements 

Charged kaon and pion candidates in the decay modes of interest must satisfy K oi n identifica- 
tion criteria [13] that are typically 85% efficient, depending on momentum and polar angle. The 
misidentification rates are at the few percent level. The invariant mass of the Ktt pair must be 

within 18.8 MeV (2.5o") of the mean reconstructed D^ mass. For modes with TT and 

7 the mass difference AM between the y*°and the b^must be within 3.5 (3.5cr) and 13 

(2(t) MeV, respectively, of the expectation for t'*'^ decays. 

A major background arises from DK and D*K decays where the K and tt in the D decay are 
misidentified as a tt and a K respectively. When this happens, the decay could be reconstructed 
as a DK or D*K signal event. To eliminate this background, we recompute the invariant mass 

(Mswitch) of the h'^h" pair in i)^— h'^h" switching the particle identification assumptions (tt vs. 
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K) on the /i+ and the h . We veto any candidates with Afs„itch consistent with the known D 
mass [14] . In the case of h^K, we also veto any candidate where the is consistent with D* decay. 

3.2 Neural Network 

After these initial requirements, backgrounds are overwhelmingly from continuum events, especially 
e+e- ^ cc, with c ^ D°X, K+tt' and c ^ DX, D R-Y . 

The continuum background is reduced by using neural network techniques. The neural network 
algorithms used for the DK and DK modes are slightly different. First, we use for both modes a 
common neural network (NN) based on nine quantities that distinguish between continuum and 
BB events. Then, for the D*K mode only, we also take advantage of the fact that the signal is 
distributed as cos^Oo* for D* Dtt or sin^ Od* for D* D7, while the background is roughly 
independent of cos^d*. Here 6d* is the decay angle of the D*, i.e., the angle between the direction 
of the D and the line of flight of the D* relative to the parent B, evaluated in the D* rest frame. 
Thus, we construct a second neural network, NN', which takes as inputs the output of NN and 
the value of cos Od* . We then use as a selection requirement the output of NN in the DK analysis 
and the output of A'^A'"' in the D*K analysis. 

The nine variables used in defining A^A^ are the following: 

1. A Fisher discriminant constructed from the quantities Lq = Y^iPi and L2 = Y^^piCOS^Oi 
calculated in the CM frame. Here, pi is the momentum and 9i is the angle with respect to the 
thrust axis of the B candidate of tracks and clusters not used to reconstruct the B meson. 

2. I cos 6t\, where 6t is the angle in the CM frame between the thrust axes of the B candidate 
and the detected remainder of the event. The distribution of | cos^tI is approximately flat 
for signal and strongly peaked at one for continuum background. 

3. cos^B, where 6b is the polar angle of the B candidate in the CM frame. In this variable, the 
signal follows a sin^ 6b distribution, while the background is approximately uniform. 

4. cos^^ where 9^ is the decay angle in i^^— ^ Ktt. 

5. cos 6^, where 6^ is the decay angle in i? — > h^K or 5 ^ y*°K. 

6. The charge difference AQ between the sum of the charges of tracks in the hemisphere 
and the sum of the charges of the tracks in the opposite hemisphere excluding the tracks used 
in the reconstructed B. For signal, (AQ) = 0, whereas for the cc background (AQ) ~ | x Qb, 
where is the charge of the B candidate. The AQ RMS is 2.4. 

7. Qb ' Qk, where Qk is the sum of the charges of all kaons not in the reconstructed B. In 
many signal events, there is a charged kaon among the decay products of the other B in the 
event. The charge of this kaon tends to be highly correlated with the charge of the B. Thus, 
signal events tend to have Qb • Qk < — 1- On the other hand, most continuum events have 
no kaons outside of the reconstructed B, and therefore Qk = 0. 

8. The distance of closest approach between the bachelor track and the trajectory of the 
This is consistent with zero for signal events, but can be larger in cc events. 
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9. The existence of a lepton (e or /x) and the invariant mass {rriKt) of this lepton and the bachelor 
K. Continuum events have fewer leptons than signal events. Furthermore, a large fraction of 
leptons in cc events are from D — ^ Ktu^ where K is the bachelor kaon, so that rriKi < uid- 

The neural networks {NN and NN') are trained with simulated continuum and signal events. 
The distributions of the NN and NN' outputs for the control samples {Dtt, D*it, and off resonance 
data), are compared with expectations from the Monte Carlo simulation in Figure 2. The agreement 
is satisfactory. We have also examined the distributions of all variables used in NN and NN', and 
found good agreement between the simulation and the data control samples. 

Our final events selection requirement is NN > 0.5 for DK and NN' > 0.5 for D*K. In addi- 
tion, to reduce the remaining BB backgrounds, we also require cos 6^ > —0.75. These requirements 
are about 40% efficient on simulated signal events, and reject 98.5% of the continuum background. 
Note, however, that we do not rely on the Monte Carlo simulation to estimate the efficiency of the 
neural net requirements. We apply the exact same requirements to the normalization modes DK 
and D*K. Then, in the extraction of TZktt ^^nd 7^^^, the efficiencies of the overall selection cancels 
in the ratio. 

(*) 

3.3 Fitting for event yields and Ti-j^^ 

The ratios TZjCn and TZ^^ are extracted from the ratios of the event yields in the uies distribution 
for the signal modes {DK and D*K) and the normalization modes {DK and D*K), while taking 
into account potential differences in efficiencies and backgrounds. All events must satisfy the 
requirements discussed above and have a AE value consistent with zero within the resolution 
(-52 MeV < A£; < 44 MeV). 

The mES distributions for DK (signal mode) and DK (normalization mode) are fitted simul- 
taneously. The fit parameter TZktt is given by TZktt = c- N^j^/N^k, where N^j^ and Nj^k are the 
fitted yields of DK and DK events, and c is a correction factor, determined from Monte Carlo, for 
the ratio of efficiencies between the two modes. We find that this factor c is consistent with unity 
within the statistical accuracy of the simulation, c = 0.98 ± 0.04^. 

The distributions are modeled as the sum of a threshold combinatorial background func- 
tion [15] and a Gaussian centered at ms- The parameters of the background function for the signal 
mode are constrained by a simultaneous fit of the tues distribution for events in the sideband of AE 
(—120 MeV < AE' < 200 MeV, excluding the AE signal region defined above). The parameters of 
the Gaussian for the signal and normalization modes arc constrained to be identical. The number 
of events in the Gaussian is Ngig + Np^ak, where Ngig = N^k or Nj^j^ and Np^ak is the number of 
background events expected to be distributed in the same way as the DK or DK in ttt-es ("peaking 
backgrounds"). 

There are two classes of peaking background events: 

1. Charmless B decays, e.g., B^ — >■ K^K~'k^. These are indistinguishable from the DK signal 
if the K~iT'^ pair happens to be consistent with the D-mass. 

2. Events of the type B~ D^Tr~ {Dtt), where the bachelor tt" is misidentified as a K~. When 
the D° decays into K~7r+ (K+7r~), these events are indistinguishable in mES from DK {DK), 
since ttzes is insensitive to particle identification assumptions. 

•'In the D* modes this correction factor is c = 0.97 ± 0.05 and c = 0.99 ± 0.05 for D* DTr° and D* Dj 
respectively. 



13 



>1000, 
800- 
600;^ 
400- 
200;^ 
0-^ 



(a) D K signal 
BaBar 

preliminary 



-1 -0.5 0.5 1 

NN output 



|1600 




-1 -0.5 0.5 1 

NN output 



500 
400 
300 
200 
100 




140 
120 
100 
80 
60 
40 
20 




(b) D K(rf') signal 



V 







g 200 

> 

ta 
150 

100 

50 

Op- 



(c) D K(y) signal 



1 '^".i 



-1 -0.5 0.5 1 

NN' output 



-1 -0.5 0.5 1 

NN' output 



(e)D KCji") background 



i 



-1 -0.5 0.5 1 

NN' output 




-1 -0.5 0.5 1 

NN' output 




-0.5 0.5 1 

NN output 



0.5 1 
NN' output 



0.5 1 
NN' output 



Figure 2: Distributions of the continuum suppression neural network (A^A^ and NN') outputs 
for the three modes. Figures (a-c) show the expected distribution from signal events. The solid 
line histogram shows the distribution of simulated signal events, the histogram with error bars 
shows the distribution of D^*^^tt control sample events with background subtracted using the ttt-es 
sideband. Figures (d-i) show the expected distribution for continuum background events. The solid 
line histogram shows the distribution of simulated continuum events and the histogram with errors 
show the distribution of off-resonance events. The m-Es and AE requirements on the off-resonance 
and continuum Monte Carlo events have been kept loose to increase the statistics. Figures (g-i) 
show the distributions of figures (d-g) in log scale. Each Monte Carlo histogram is normalized to 
the area of the corresponding data histogram. 
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The amount of charmless background (1) is estimated directly from the data by performing a 
simultaneous fit to events in the sideband of the reconstructed D mass. The distribution of 
the D-K background (2) is shifted by about +50 MeV due to the misidentification of the bachelor 
as a K. Since the A£' resolution is of order 20 MeV, the /S.E requirement docs not eliminate 
this background completely. The remaining Dtt background after the A.E requirement is estimated 
from a fit to the A£' distribution of the DK sample. 

We fit the /S.E distribution of DK candidates, with mES within Scr of m^, to the sum of a DK 
component, a Dtt component, and a combinatorial component. The Dtt sample, with the bachelor 
track identified as a pion, is used to constrain the shape of the DK component in the DK sample. 
The same sample of Dtt events, but reconstructed as DK events, is used to constrain the shape 
of the Dtt background in the DK sample. The fitted number of Dtt background events in this 
sample that survive the /S.E requirements, which we denote as N'^j^, is taken as the number of Dtt 
background events in the fit to the uies distribution of DK events.. 

The Dtt peaking background is much more important in the DK (normalization) channel than 
in the DK (signal) channel. This is because in order to contribute to the signal channel, the 
D^ has to decay into K~^7r~, and this mode is doubly Cabibbo-supprcsscd. For the DK (signal) 
sample, the contribution from the residual Dtt peaking background in the uies fit is estimated as 
N'^^ = r'jjN^^, where td = 0.060 ± 0.003 is the ratio of the doubly Cabibbo-suppressed to the 
Cabibbo- favored D Kir amplitudes (see Eq. 6), and N'^j^ was defined above. 

The complete procedure simultaneously fits seven distributions: the niES distributions of DK 
and DK, the DK distributions in sidebands of IS.E and m{D^), the /S.E distribution of DK, and 
the A£' distributions of Dt: reconstructed as Dvr and as DK. The fits are configured in such a 
way that TIk-k and Tlx-n are explicit fit parameters. The advantage of this approach is that all 
uncertainties, including the uncertainties in the PDFs and the uncertainties in the background 
subtractions, are automatically correctly propagated in the statistical uncertainty reported by the 
fit. 

The fit is performed separately for DK, D*K, D* Dtt^, and D*K, D* Dj and is identical 
for all three modes, except in the choice of parameterization for some signal and background 
components in the AE fits. 

Systematic uncertainties in the detector efficiency cancel in the ratio. This cancellation has 
been verified by studies of simulated events, with a statistical precision of a few per-cent. The 
likelihood includes a Gaussian uncertainty term for this cancellation which is set by the statistical 
accuracy of the simulation. Other systematic uncertainties, e.g., the uncertainty in the parameter 
r£) which is used to estimate the amount of peaking backgrounds from D^*\, are also included in 
the formulation of the likelihood. 

The fit procedure has been extensively tested on sets of simulated events. It was found to 
provide an unbiased estimation of the parameters TZ/Cn and ^Jctt- 

4 RESULTS 

The results of the fits are displayed in Table 2 and Figs. 3, 4, 5, and 6. As is apparent from Fig. 6, 
we see no evidence for the D*K modes and no significant evidence for the DK mode. 

For the DK mode we find TZicn = ISlg^ x lO'^; For the D*K mode we find TZ*^^ = -1+^° x lO'^ 
(for D* D7r°) and TZ'^^ = x 10~^ (for D* D^). We estimate from a parameterized 

Monte Carlo study that the probability that an upward fluctuation of background events results in 
our observed value of TZic-k or larger is 7.5%. 
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A E (GeV) A E (GeV) A E (GeV) 

Figure 3: AE distributions for normalization events (DK and D*K) with rngs within 3(7 of 
with the fit model overlaid, (a) DK events, (b) D*K events with D* Dir^. (c) D*K events 
with D* — > D'j. The dashed (dot-dashed) curves are the contributions from Dtt or D*tt (DK or 
D*K) events. The dotted curves are the contributions from other backgrounds, and the solid line 
is the total. 



Table 2: Summary of fit results. 



Mode 


DK 


D*K, D* Dtt^ 


D*K, D* D-f 


Ratio of rates, T^-^Ctt or 1^*^^^-, xlO"^ 

No. of signal events 

No. of normalization events 

No. of peaking charmless events 

No. of peaking D*^*^7r ev. in sig. sample 

No. of peaking D^*^Tr ev. in norm, sample 


T^Ktt = 13+9^ 
4 7+4-0 

^- ' -3.2 

356 ± 26 

o.75lJ:?5 

0.47 ±0.04 
132 ± 10 


'!-)* -1 +iU 

'■^/Ctt — "J- -6 

-0 2+1-3 
142 ± 17 

n+0.3 

•-"-o.o 

0.17 ±0.02 

48 ±6 


'%7r — 11^-13 

1 2+2-1 

■■-•^-1.4 

101 ± 14 
0-05+°5^ 

o.oiir' 

2.5 ±8 



We use our results to extract information on and r^. In the case of decays into a D* / D* we 
can use equations 11 and 12 to write 

^g^ ^w±7^|,.,,,, _^^^ (13) 

Our results then give r*^ = 4.6+7^3^ x 10"^. The likelihood function for r*^ is shown in Figure 7. 
Based on this likelihood, we set an upper limit r*^ < (0.16)^ at the 90% C.L. using a Baysean 
method with a uniform prior for r"^ > 0. 

In the case of decays into a D/D, there are not enough information to extract the ratio rs 
without additional assumptions. Thus, we first extract an upper limit on the experimentally mea- 
sured quantity TZiCn ■ This is done starting from the likelihood as a function of TZjctt (see Figure 8) 
using a Bayesian method with a uniform prior for TZiCn > 0. The limit is TZjCn < 0.030 at 90% 
C.L.. Next, in Fig. 9 we show the dependence of TZ^n on r^, together with our limit on TZk-k- 
This dependence is shown allowing a ±1(t variation on r/j, for the full range 0° — 180° for 7 and 
(5, as well as with the restriction 48° < 7 < 73° suggested by global CKM fits [16]. We use the 



16 




(GeV) 



Figure 4: rriES distributions for normalization events (DK and D*K) with IS.E in the signal region 
with the fit model overlaid, (a) DK events, (b) D*K events with D* Dtt^. (c) D*K events 
with D* — > D'j. The dashed curves represent the backgrounds; these are mostly from Dn or D*Tr, 
and also peak at the i?-mass. As explained in the text, the size of the Dir and D*tt backgrounds 
is constrained by the simultaneous fits to the distributions of Figure 3. 



information displayed in this Figure to set an upper limit on r^. The least restrictive limit on 
is computed assuming maximal destructive interference: 7 = 0°, (5 = 180° or 7 = 180°, 6 = 0°. The 
limit is rij < 0.23 at 90% C.L.. 
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Figure 5: mES distributions for DK and D*K events with Kir mass in a sideband of the recon- 
structed D mass and with AE in the signal region. These events are used to constrain the size 
of possible peaking backgrounds from charmless S-meson decays, i.e., decays without a D in the 
final state. The fit model is overlaid, (a) DK events, (b) D*K events with D* Dtt^. (c) D*K 
events with D* — > Dj. Note that the Kir mass range in the sideband selection is a factor of 2.7 
larger than in the signal selection. 
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Figure 6: mss distributions for candidate signal events with the fit model overlaid, (a) DK events, 
(b) D*K events with D* Dtt^. (c) D*K events with D* D-f. 
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Figure 9: Expectations for TZxn and the number of signal events vs. rs- Dark filled-in area: allowed 
region for any value of 6, with a ztla variation on r^), and 48° < 7 < 73°. Hatched area: additional 
allowed regions with no constraint on 7. Note that the uncertainty on r^) has a very small effect 
on the size of the allowed regions. The horizontal line represents the 90% C.L. limit TZk-k < 0.030. 
The vertical dashed lines are drawn at = 0.209, rs = 0.235. They represent the 90% C.L. upper 
limits on rs with and without the constraint on 7. The light filled in areas represent the 68% C.L. 
regions corresponding to TZr-k = 0-013±g;oQg. 



5 SUMMARY 

In summary, we find no significant evidence for the decays — > [K^tt^]dK^ and — > 
[K^7r^]£)*K^. We measure the ratios TZk-k of the rates for these modes and the favored modes 
B± ^ [K^Tr^]DK^ and 5± ^ [K^7r^]D*K^ as TZxn = 0.013±[]:[|J^, 71}^^ ^^o = -0.001±[]:[|J|] and 

7^|,,,^^ = o.oll±g:0{l. 

We use the results for T^*j^^ ij^o and TIxt^ D'j *o a model independent limit r'^ = \A{B~ — > 
D^K-)/A{B- Z)°i^-)|2 < (0.16)2 at the 90% confidence level. We also set 90% C.L. limits on 
the ratio TZk-k < 0.030 (90% C.L.). With the most conservative assumption on the values of 7 
and of the strong phases in the B and D decays, this limit translates into a limit on the ratio of 
the magnitudes of the B~ D^K~ and B~ — > D^K~ amplitudes amplitude < 0.23 at 90% 
C.L.. If tb and r*^ are small, as our analysis suggests, the suppression of the h ^ u amplitude will 
make the determination of 7 using methods based on the interference of the diagrams in Figure 1 
difficult. 
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